Artificial Intelligence for Evidence Synthesis of Emerging Biologics to Improve Skeletal Health in Osteogenesis Imperfecta: Systematic Review and Meta-Analysis

Introduction

Osteogenesis imperfecta (OI) is a rare genetic connective tissue disorder with an estimated incidence of 1 in 15,000 to 20,000 live births [1]. It is characterized not only by bone fragility, recurrent fractures, and bone deformities, but also by growth impairment, chronic bone pain, and hearing loss, all of which contribute to lifelong disability and reduced health-related quality of life [2]. OI is primarily caused by mutations affecting type I collagen, the principal structural protein in bone [3], resulting in an abnormal collagen matrix and structurally fragile bone [4]. To date, more than 22 subtypes have been identified, with type I OI being the most common and clinically mild, whereas other subtypes (eg, types II-IV) are more severe [5,6].

Current therapeutic strategies for OI remain palliative, focusing only on symptomatic relief. Bisphosphonates (BPs), such as alendronate, zoledronate, and neridronate, are small-molecule antiresorptive agents that increase bone mass by nonselectively binding to bone surface hydroxyapatite and inhibiting osteoclast-mediated bone resorption. For the past two decades, they have been widely recommended and routinely used as first-line therapy for OI, particularly in children with moderate-to-severe disease. However, several systematic reviews of randomized controlled trials (RCTs) have shown that they are less effective in adults or patients with severe phenotypes (eg, types II-IV) [7-9] and are associated with common side effects such as upper gastrointestinal irritation and acute-phase reactions [10].

In parallel, emerging biologics—such as denosumab (RANKL inhibition) [11], setrusumab and romosozumab (sclerostin neutralization) [5,12], teriparatide (parathyroid hormone activation) [13], and fresolimumab (TGF-β inhibition) [14]—provide more targeted mechanisms of action by modulating specific molecular pathways in bone remodeling. These biologics offer potential for improved efficacy and fewer off-target side effects compared to BPs. But their clinical value remains uncertain: most data come from early-phase single-arm studies or small randomized trials, with heterogeneous follow-up durations and inconsistent findings for functional status and fracture outcomes. Existing syntheses are limited by narrow drug scope, age-restricted samples, and variability in study design. Few have quantitatively pooled safety and efficacy data across biologic classes, and none, to our knowledge, have systematically evaluated biologics in both adults and children with OI using meta-analytic methods. This evolving yet fragmented evidence base underscores the need for an up-to-date synthesis to inform clinical decision-making.

However, conducting such comprehensive reviews in OI is challenging. Heterogeneous study designs and inconsistent outcomes make manual screening and appraisal labor-intensive and prone to cognitive fatigue [15,16]. Advances in large language models (LLMs) offer promising avenues to reduce human workload by automating parts of the systematic review workflow, but current applications are limited in both task scope and methodological rigor. Prior studies have primarily focused on narrow aspects such as Boolean query optimization [17] or abstract-level screening [18]. GPT-4o has been applied to full-text analysis [19], yet artificial intelligence (AI)–driven quality appraisal was omitted. Beyond these technical limitations, existing implementations may introduce human-AI interaction biases; for example, automation bias may lead reviewers to over-trust model-generated inclusion decisions [20,21], and conversely, a conventional “human-in-the-loop” framework can reproduce human biases and further compromise the generalizability of the model [22,23]. Collectively, these limitations call for a more comprehensive evaluation of LLMs in evidence synthesis—one that goes beyond isolated tasks and combines human expertise with automation in a way that is reliable and generalizable to real-world review workflows.

Therefore, we conducted a systematic review and meta-analysis of clinical trials of biologics for OI to evaluate their efficacy and safety. To support the review process, we also implemented an LLM-assisted workflow that simulates integrated human screening and appraisal, and evaluated its agreement with consensus decisions by human reviewers.

Methods

We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 reporting guideline and referred to the PRISMA 2020 expanded checklist for detailed explanations (Checklist 1) [24].

Search Strategy and Eligibility Criteria

To first identify novel biologics under investigation for the treatment of OI, we horizontally scanned ScanMedicine and ClinicalTrials.gov for all phases of clinical trials registered up to December 1, 2025. Trials were excluded if they were withdrawn, suspended, or conducted solely in animal models. Only those reporting preliminary efficacy outcomes were included. Through this process, we identified 7 biologics: teriparatide, fresolimumab, setrusumab, romosozumab, blosozumab, somatropin, and denosumab.

To further evaluate the clinical efficacy of biologics, we systematically searched PubMed, Embase, ScienceDirect, Web of Science, and the Cochrane Library for human studies published up to December 1, 2025. All databases and registries were searched separately, not via a multidatabase platform. No additional trial registries (eg, WHO ICTRP) were searched. The search terms combined free-text in titles, abstracts, and keywords: (“osteogenesis imperfecta” OR “fragilitas ossium” OR “dysostosis” OR “osteopsathyrosis” OR “brittle bone disease”) AND (“teriparatide” OR “fresolimumab” OR “setrusumab” OR “BPS804” OR “romosozumab” OR “blosozumab” OR “TST002” OR “somatropin” OR “denosumab”). Because search syntax differs across interfaces, the full database-specific strategies (including field tags/modifiers) are reported in Table S1 in Multimedia Appendix 1. Additionally, ClinicalTrials.gov was searched for relevant ongoing or completed but unpublished trials by entering “osteogenesis imperfecta” in the Condition or Disease field and the investigational drug name in the Intervention/Treatment field.

We also undertook supplementary browsing of conference proceedings and pharmaceutical press releases to supplement outcome data when original trial results were not available. In addition, we hand-searched the reference lists of included trials for additional eligible articles and contacted trial investigators when necessary to obtain missing information, such as standard deviations. Forward citation searching using citation indexes was not conducted. No published search filters were applied. Searches were run once, with restriction to English-language publications and a last update on December 1, 2025, for all databases; no study-design limits were otherwise imposed. The search strategy was developed de novo, was not adapted from previous reviews, and was formally peer-reviewed by a third information specialist.

Table S1 in Multimedia Appendix 1 also reports the number of records retrieved from each database and the total number of records before and after deduplication in EndNote (Clarivate). All records were imported into EndNote 21 for automated deduplication and then exported to Microsoft Excel for additional manual removal of duplicates; therefore, post-deduplication counts are only available for the overall dataset, not for each individual database. The literature search and their reporting followed the PRISMA-S checklist (Checklist 2) [25].

Selection Process

Manual Screening

Studies were eligible for inclusion if they (1) were RCTs, non-RCTs, quasi-randomized trials, crossover trials, prospective interventional open-label trials (including single-arm studies), or historical-control interventional trials; (2) reported the efficacy outcomes of biologics identified from the horizontal scanning; and (3) provided a well-established diagnosis and subtypes of OI. Studies were excluded if they (1) were observational studies, case reports, books, or reviews; (2) investigated interventions that combined the biologics of interest with BPs or other nonlisted agents which could confound treatment effects; (3) recruited patients with impaired renal function, liver disease, hypocalcemia, or other comorbidities affecting bone metabolism, except for postmenopausal osteoporosis; and (4) did not report quantified results, especially changes in areal bone mineral density (aBMD) or fracture incidence. Detailed criteria, including the general criteria used at the title/abstract screening stage and the more specific outcome-based criteria used at the full-text screening stage, are provided in Table S2 in Multimedia Appendix 1. Two reviewers independently screened titles and abstracts for eligibility and retrieved full texts where applicable. Discrepancies were resolved through discussion until a consensus was reached.

LLM-Assisted Screening

To evaluate the screening capabilities of LLMs, a screening tool powered by the GPT-4o model via the Azure OpenAI API was used. Inclusion and exclusion criteria were encoded into zero-shot structured prompts. To mirror manual procedures, the process was conducted in two stages: title/abstract screening and full-text screening. Before initiating full-text screening, studies without accessible full-text documents were excluded, and only available full-text PDFs were provided to the model. During the first stage, the model categorized each study into 1 of 3 levels of relevance: “yes” (included), “no” (excluded), and “maybe” (potentially included). Studies classified as “yes” or “maybe” at the initial stage were automatically advanced to full-text screening for further evaluation. The performance of the LLM-assisted tool was benchmarked against the results of manual review, using sensitivity (recall) and specificity as the primary metrics. To facilitate reproducibility, the zero-shot structured prompts for both title/abstract and full-text screening, along with the output schema, are provided in Multimedia Appendix 2.

Data Collection and Outcomes

Two human reviewers independently performed data extraction using a standardized form, and only articles that passed human screening were included. Extracted data included (1) innovative medicine; (2) first author and publication year; (3) study design; (4) follow-up period; (5) types of OI; (6) methods of diagnosis; (7) participants (gender and age); (8) sample size and intervention (per group); (9) primary efficacy outcomes (changes in bone mineral density [BMD], fracture incidence, functional or pain scores, and other prespecified clinical endpoints); and (10) secondary safety outcomes (adverse events, serious adverse events, and mortality). If the required data were missing, attempts were made to contact the authors to request the additional information for clarification.

Risk of Bias Assessment

Manual Assessment

The quality of clinical trials was assessed using the revised Cochrane risk-of-bias tool for randomized trials (RoB 2) [26]. The tool evaluates risk of bias across 5 domains through 22 signaling questions, and each item was rated as yes (Y), probably yes (PY), probably no (PN), no (N), no information (NI), or not available (NA). For single-arm trials without control groups, specific RoB 2 items were modified by refining comparator-related domains and adjusting criteria for randomization and allocation. Two reviewers independently conducted the assessment, with disagreements resolved in consensus. Detailed risk of bias assessments, including the modified criteria for single-arm trials, are presented in Section S3 in Multimedia Appendix 1.

LLM-Based Assessment

An LLM-assisted approach was used to streamline the risk of bias assessment. Customized RoB 2 prompts, aligned with original guidelines, were integrated into the GPT-4o API to systematically analyze full-text PDFs selected by humans and generate standardized judgments for all signaling questions. Details of the RoB 2 prompts, including adapted signaling questions, response options, and parsing instructions, are provided in Multimedia Appendix 3.

To evaluate the interrater reliability between human raters and the GPT-4o, we calculated weighted Cohen κ coefficients for each paper, applying 5 ordinal rating categories [27]. Kappa values were computed in IBM SPSS Statistics, with linear weights of 1, 0.75, 0.5, 0.25, and 0 assigned to Y, PY, NI, PN, and N, respectively. NA, coded as 999 in the dataset, were treated as missing values in the analysis. The overall weighted κ coefficient was derived by aggregating article-level κ values using the inverse-variance weighting method. Consistent with widely accepted benchmarks, a global weighted κ coefficient of 0.6 or higher was classified as substantial agreement, reflecting high interrater consistency [27].

Data Synthesis

For drugs with at least two trials, meta-analyses of efficacy outcomes were performed using the meta package in R (version 4.4.3), applying a random-effects model with the Hartung-Knapp-Sidik-Jonkman (HKSJ) method to account for between-study heterogeneity [28]. The primary efficacy outcome was the BMD z score, percentage change in BMD from baseline, and fracture incidence. For the main analyses, we conducted single-arm pre-post meta-analyses by pooling only the innovative-treatment arms from each eligible trial, irrespective of whether the original study was randomized or nonrandomized. Comparative effectiveness, when available, is reported separately in Table S4 in Multimedia Appendix 1 [14,29-39]. Each clinical trial contributed only a single effect estimate to each pooled analysis, thereby avoiding any double-counting of trials.

In accordance with Cochrane guidelines, publication bias was not formally assessed due to the limited number of studies (<10 per analysis), thus precluding the use of the Egger regression or funnel plot asymmetry tests. To address age-related heterogeneity, we first performed a primary meta-analysis at the all-age level, including all eligible trials, then conducted subgroup analyses stratified by age group (adults vs children). Additionally, to examine the impact of original study design, we conducted sensitivity analyses stratified by study design, pooling (1) intervention arms from randomized trials separately from (2) nonrandomized single-arm/self-controlled studies; results are presented in Figure S1 (a-e) in Multimedia Appendix 1. Results for drugs that were not eligible for pooling with fewer than two trials or only qualitative, nonpooled efficacy data are narratively summarized in Section S4a in Multimedia Appendix 1.

Results

Study Selection: GPT-4o–Assisted Screening vs Human Reviewers

The comparison between manual and GPT-based screening is summarized in Figure 1. Both approaches started from the same pool of 594 records, with 294 unique records screened after deduplication. GPT-assisted screening was applied at the report level and classified 52 full-text reports as eligible following title/abstract and full-text assessment, whereas manual screening ultimately included 18 eligible full-text reports from the same pool. Because multiple reports could originate from the same underlying trial, all eligible full-text reports were subsequently mapped to unique studies in a separate, manual study-level consolidation step, in which multiple reports of the same trial were merged. After this consolidation, 13 unique studies were included in the systematic review [14,29-40], of which 10 contributed data to the meta-analysis [29,30,32-34,36-40].

‎

Notably, GPT-based screening dramatically reduced workload, with an average screening time of 3.5 (SD 0.14) seconds per article for title and abstract screening and 9.67 (SD 1.85) seconds for full-text review (Figure S2 in Multimedia Appendix 1), compared to approximately 15 minutes per article for manual screening. This resulted in over 95% total time savings and an approximately 100-fold increase in screening speed per article.

Using manual review as the benchmark, GPT achieved high sensitivity—97.4% for title/abstract screening and 88.9% for full-text review—but lower specificity, at 77% and 87%, respectively, indicating a tendency to favor recall over specificity (Table 1).

Characteristics of Studies for Systematic Review

A total of 13 trials were included (684 participants) [14,29-40]: 8 controlled studies (4 double-blind trials [29,33,35,39], 2 open-label trials with zoledronate as active comparator [36,37], 1 phase I trial [14], and 1 historical-controlled study [38]), and 5 uncontrolled trials (3 single-arm trials [30,31,34] and 2 open-label, self-controlled trials [32,40]). Interventions included denosumab (n=6), teriparatide (n=3), fresolimumab (n=1), romosozumab (n=1), and setrusumab (n=2), with comparators such as placebo or bisphosphonates. The studies enrolled 355 adults and 329 children and reported outcomes included changes in areal or volumetric BMD, BMD z score, fracture incidence, bone turnover markers, and safety profiles. Study characteristics are presented in Table 2, with efficacy outcomes detailed in Table S2 in Multimedia Appendix 1.

Risk of Bias: GPT-4o–Assisted Assessment vs Human Ratings in Studies Included in Meta-Analysis

Manual risk of bias in 10 trials is presented in Figure S3 in Multimedia Appendix 1. Four trials [30,33,34,40] were rated as low risk across all domains. Five trials exhibited a high risk of bias [32,36-39], mainly due to subjective assessment of spinal morphometry in outcome measurement (n=4) [32,36-38], or missing values that could have biased the outcome measurements (n=1) [39]. The overall risk of bias was adjudicated as some concerns for all 10 studies.

Table S1 in Multimedia Appendix 1 compares the risk of bias between human evaluators and the GPT-based tool for the 10 studies. Overall, substantial agreement was observed (weighted κ=0.778, 95% CI 0.710‐0.846) with no significant heterogeneity across studies. All studies except Glorieux et al [33] and Mei et al [38] exceeded the 0.6 threshold for substantial agreement. The greatest discrepancies between human and GPT evaluations were found in the domains of randomization, followed by measurement of the outcome.

Efficacy of Biologics

Age-Adjusted BMD Z-Score Change at Lumbar Spine

Denosumab

Three trials [30,34,37] in children reported significant lumbar spine z score increases with denosumab at 6 and 12 months. However, the largest existing RCT [37] found no significant advantage of denosumab over zoledronic acid after 12 months (P>.05), devaluing the results of the smaller retrospective analysis in Hoyer-Kuhn et al [34] study (n=8), which had reported a greater z score gain with denosumab than prior bisphosphonate therapy (mean increase: 1.15 vs 0.31, P=.02). A pooled analysis using a meta-analytic approach (though not a formal meta-analysis) of data from Hoyer-Kuhn et al [34] and Amgen Inc [30] yielded a weighted mean z score increase of 1 (95% CI 0.811-1.195; n=55) at 12 months.

Setrusumab

In the single-arm phase 2 of the ORBIT study (ages 5‐26 years) [29], setrusumab produced significant improvements in lumbar spine BMD z scores from baseline (P<.001). Pooled across the 20 mg/kg and 40 mg/kg dose groups, mean z score increases were 0.85 (SE 0.13) at month 6 and 1.25 (SE 0.17) at month 12, indicating a robust early response. A longer follow-up is needed to determine sustained effects in z score.

Area BMD Percentage Change at Lumbar Spine

Twelve trials reported percentage change in lumbar spine aBMD [14,29,31-40], of which 9 trials (180 participants) provided sufficient data for inclusion in the formal meta-analysis (Figure 2) [29,32-34,36-40]. A random-effects model using the HKSJ method was applied, given the clinical and methodological diversity across trials. The pooled percentage increase was 12.93% (95% CI 5.4%‐20.45%). Of the 9 pooled studies, 5 were at high overall risk of bias, mainly due to open-label designs and nonblinded aBMD assessment [32,36-39], potentially introducing subjective measurement influences and additional uncertainty around the pooled effect. Between-study heterogeneity was substantial (I²=88.6%, τ²=37.10). Subgroup analyses by drug showed statistically significant differences between treatments (test for subgroup differences: χ²₂=9.2, P=.01). Denosumab at 12 months was associated with the largest mean increase (18.76%, 95% CI 5.44%‐32.08%; I²=88.7%), whereas teriparatide at 18 months showed a smaller and statistically nonsignificant increase (5.81%, 95% CI −4.66% to 16.27%; I²=0%). Setrusumab at 12 months showed a mean increase of 15.61%, but with a very wide and imprecise confidence interval (95% CI −66.11% to 97.34%; I²=94.3%), indicating considerable uncertainty about the true treatment effect.

‎

Subgroup analysis showed that in children (Figure 3), the pooled percentage increase in lumbar spine aBMD across the two biologics was 23.54% (95% CI 18.99%‐28.08%; I²=0%, τ²=0). Denosumab at 12 months was associated with a mean increase of 25.49% (95% CI 17.14%‐33.84%; I²=0%; τ²=0) and setrusumab at 12 months showed a similar effect (22.25%, 95% CI 16.94%‐27.56%; τ²=0). The difference between the two biologics was not statistically significant (test for subgroup differences: χ²₂=0.7, P=.39).

‎

In adults (Figure 4), the pooled percentage increase in aBMD across the three biologics was 6.1% (95% CI 3.13%‐9.06%; I²=47.2%, τ²=2.53), indicating moderate between-study heterogeneity. There was evidence of differences between agents (test for subgroup differences: χ²₂=14.3, P<.001). Setrusumab at 12 months produced the largest effect (9.38%, 95% CI 6.5%‐12.26%). In contrast, denosumab at 12 months (4.29%, 95% CI 1.27%‐7.31%; I²=0%) and teriparatide at 18 months (5.81%, 95% CI −4.66% to 16.27%) showed smaller and, for teriparatide, statistically uncertain effects. Notably, the setrusumab trial was judged to be at low risk of bias, whereas 4 trials evaluating denosumab and teriparatide were at high risk of bias, adding further uncertainty to these estimates.

‎

Fracture Reduction

Evidence on fracture reduction with biologics in OI remains limited and mixed. Across 3 trials [30,36,37] and 1 interventional study with historical control [38], denosumab demonstrated fracture rates comparable to bisphosphonates (zoledronic acid and alendronate) and in both adults and children. Intensifying the dosing frequency to every 3 months in children did not yield a lower fracture rate (Q3M: 26.7% vs Q6M: 28.3%; P>.05) [30]. For teriparatide, 2 adult trials [35,39] found teriparatide (20 μg/day) reduced fracture risk compared to controls, although results were not statistically significant. New fracture rates were lower with teriparatide versus placebo (29% vs 36%, odds ratio 0.73) over 18 months [39] and intravenous neridronate over 24 months (16.3% vs 26.5%; P=.10) [35]. A single trial with setrusumab showed dose-dependent fracture reduction [33]: the 20 mg/kg group had the lowest fracture incidence (16.1%) and annualized fracture rate (0.19 per participant-year), compared to 34.5% and 0.54 in the 8 mg/kg group over 12 months.

Safety of Biologics

Ten studies reported adverse events as safety outcomes [14,29-34,36-38]. Selective outcomes are summarized in Table 3, while all reported outcomes are listed in Table S5 in Multimedia Appendix 1. Metabolic disturbances were notable, with denosumab associated with a high incidence of hypercalcemia (up to 30.95%) and hypercalcemic crisis (14.29%) in children [37]. In adults, denosumab did not increase the risk of hypercalcemia compared to zoledronic acid and significantly reduced the risk of acute-phase reactions (P=.002) [36]. Teriparatide showed the highest rate of nausea among adults (53.85%) [32], but serious adverse event rates were similar to placebo or neridronate in head-to-head trials [35,39]. Fresolimumab was associated with medication-related adverse events in all 8 patients with type III OI in a single-arm trial, most commonly epistaxis, though none were classified as serious [14]. Setrusumab emerged as a relatively safe biologic, with no cases of hypercalcemia or treatment-related serious adverse events. Reported adverse events in adults included transient hypocalcemia and a mild headache [33]. Phase 2 trial also confirmed acceptable tolerability in young patients [29]. In a phase 1 pediatric trial of romosozumab, 48% of participants experienced mostly mild-to-moderate treatment-emergent adverse events (TEAEs), with two serious TEAEs (femur and lower-limb fractures) deemed unrelated to treatment [31].

Discussion

Principal Findings

This study addressed the clinical challenges of synthesizing fragmented trial evidence in a rare disease—OI. Clinically, among emerging biologic therapies, denosumab and setrusumab could potentially improve aBMD across age groups, but comparative evidence remains limited and is not yet robust enough to demonstrate clear efficacy superiority or fracture benefits over traditional BPs. To streamline the review process, we also implemented an LLM-assisted, automation-enabled, human-in-the-loop workflow for key steps, including screening and risk of bias assessment. Its outputs showed acceptable agreement with consensus human decisions.

Novel Biologics for OI: Efficacy and Uncertainty

Patients with OI face substantial unmet needs, with fracture reduction and improved quality of life as central goals [41]. However, most existing studies use changes in aBMD as a surrogate outcome rather than directly assessing clinically meaningful endpoints such as fracture incidence, mobility, and patient-reported measures [42,43]. This is important because a higher aBMD does not always equate to stronger bones in OI [44]. Against this backdrop, our review is the first to systematically evaluate emerging biologics in OI, synthesizing age-specific advantages while underscoring the shared limitations of relying on surrogate endpoints.

In pediatric populations, both denosumab and setrusumab significantly improved lumbar spine aBMD, with denosumab showing the highest overall gain over 12 months. Nevertheless, the largest RCT has not demonstrated a significant advantage of denosumab over intravenous BPs [37], the current standard of care. For setrusumab, early improvements were observed within 12 months, but long-term efficacy and safety remain unconfirmed, as the COSMIC trial (NCT05768854) [45]—the only ongoing phase 3 RCT directly comparing setrusumab and BPs in children over a 24-month period—remains under recruitment. Evidence for fresolimumab in children is lacking, and teriparatide is contraindicated in children due to the acknowledged risk of osteosarcoma [46]. These findings suggest that, while biologics show potential in children, they are not validated for routine use and do not surpass established therapies.

In adults, setrusumab exhibited the greatest increase in lumbar spine aBMD, with higher doses yielding greater gains. This dose-response relationship may be explained by the mechanism proposed by Hosseinitabatabaei et al [47], whereby higher doses stimulate modeling-based bone formation by increasing bone mass directly without requiring prior resorption. This suggests that higher dosing may benefit patients requiring rapid bone accrual. Response to anabolic agents appears subtype-dependent: teriparatide showed efficacy mainly in type I OI [39], with limited benefit in types III and IV, while fresolimumab was effective only in type IV [14], and associated with decreased aBMD in types III and VIII. This means in the adult population, biologics should be considered primarily in the context of clinical trials, with treatment tailored to patient-specific risk profiles and OI subtypes.

Despite aBMD gains with denosumab and setrusumab across age groups, evidence for fracture risk reduction is limited. Denosumab has not shown a significant lower risk in fracture rates versus BPs, even with intensified dosing, which contrasts with findings in osteoporosis [48,49]. This disparity reflects fundamental differences in pathophysiology: in osteoporosis, increased osteoclast activity drives fragility, while in OI, it stems from genetic mutations impairing collagen synthesis. As El-Gazzar et al [50] noted, denosumab reduces bone resorption but does not improve collagen production, folding, or mineralization. Consequently, mineralized material accumulates unevenly within an abnormal collagen matrix, leading to an “over-mineralized but fragile” matrix. Similarly, Rummler et al [51] observed that setrusumab increased bone mass in OI but failed to alter the bone’s multiscale structure, suggesting that fragility arises from other unexplored aspects of bone organization at higher length scales. Also, evidence directly linking specific aBMD changes to fracture risk in OI remains inconclusive. A recent prospective OI cohort did find that lower baseline aBMD (z score < −2 SD) predicts a higher fracture risk, but genotype (COL1 splicing/stop/frameshift variants) exerted an even stronger effect, indicating that bone quality and collagen defects may modify the aBMD-fracture relationship [52].

These findings highlight the limitations of aBMD as a surrogate endpoint in OI. Clinically, what matters most to patients is whether biologics reduce fracture rates, improve mobility, and alleviate pain. Future RCTs should therefore prioritize comprehensive bone quality markers—such as the trabecular bone score and microstructural imaging—alongside direct measures of fracture reduction and patient-centered outcomes.

LLMs in Evidence Synthesis: Reading and Reasoning

Literature Screening

This study supports the use of LLMs as effective first-pass screeners to reduce reviewer burden even without domain-specific fine-tuning. During screening, GPT-4o was calibrated to prioritize sensitivity, suggesting that LLMs can substantially ease the initial workload by flagging potentially eligible studies despite complex and heterogeneous clinical trial designs—an approach that is often desirable in rare-disease reviews, where missing a key trial is costly.

However, specificity was relatively low during title and abstract screening, reflecting a conservative approach to limited or incomplete information. GPT-4o was instructed to include studies marked as “yes” or “maybe,” with “maybe” indicating insufficient information for confident exclusion. This over-inclusive approach at the initial stage, with strict exclusion deferred to full-text review, mirrors established logic in human screening workflows [53].

This “high-recall, low-specificity” pattern is consistent with the metrics reported by Dennstädt et al [54], where several openly available LLMs achieved ≥95%‐100% sensitivity but often <30% specificity when lenient thresholds were chosen to minimize false negatives. Our screening setup operated in the same sensitivity-oriented regime: GPT-4o over-included 96 studies in the first round, and it successfully identified all 13 human final-selected studies. Thus, the lower specificity is better viewed as a precautionary trade-off that prioritizes recall in the face of ambiguity.

In the full-text phase, GPT-4o produced only one false negative by erroneously excluding an eligible open-label, single-arm trial. Despite inclusion criteria explicitly permitting such designs, the model inferred—based on surrounding context—that open-label studies should involve two arms like RCTs. This reflects a pattern-based generalization bias [55], whereby the model applies common associations from its training data to oversimplify text instead of strictly adhering to the stated criteria. This highlights the importance of precise, detailed instructions tailored to specific review needs—such as the types of trial included—to help the model interpret nuanced criteria accurately.

Risk of Bias Assessment

GPT-4o achieved substantial agreement with human raters in risk assessment, but performance varied across domains, reflecting differences in the cognitive demands of each task. In the randomization domain, the model frequently inferred “yes” based on indirect contextual hints—such as a study being labeled “prospective RCT” or conducted at a well-known hospital—even when allocation concealment was not described. This reflects an optimism bias [56], whereby the model systematically overestimates the value despite lacking concrete evidence. In the measurement domain, a positional (salience) bias was observed [57]: the model assigned a low risk of bias whenever any objective outcome (eg, laboratory tests) was reported, even if subjective outcomes or lack of assessor blinding were also present. This refers to a tendency to overweight prominent or easily identifiable features while overlooking less visible but equally important factors.

Although these biases differ across tasks—overgeneralizing exclusion during full-text screening and over-crediting judging during risk assessment—they share a common root: LLMs rely on probabilistic heuristics derived from linguistic patterns rather than structured, context-sensitive reasoning [58]. As a result, current models excel at information extraction but struggle with tasks that require skepticism and uncertainty handling. This limitation becomes particularly evident in scenarios where missing information must be interpreted cautiously [59].

Looking ahead, we expect future models to move beyond better reading toward better reasoning—questioning unclear points before operation, closely adhering to review standards, and aligning more tightly with domain specifics across clinical subfields to reduce overgeneralization. From a user perspective, prompt engineering (eg, structured templates and explicit decision rules) can help mitigate bias. In practice, we recommend a hybrid approach: LLMs should assist, but not replace, human judgment, especially in tasks like full-text screening and quality assessment, where structured skepticism and contextual discernment remain essential.

Limitations

This study has several limitations. First, the limited number of eligible trials, particularly for romosozumab and fresolimumab, precluded pooled analyses and publication bias assessments. Second, the short duration of included studies (≤24 months) limits insights into long-term efficacy, safety, and rebound effects, which is crucial given OI typically necessitates lifelong pharmacological treatment. Third, clinical heterogeneity from variations in OI subtypes, doses, and prior BP use, together with substantial design heterogeneity and limited comparative data, complicates quantitative cross-trial comparisons and precludes a formal certainty-of-evidence assessment (eg, GRADE). Fourth, all prompts were administered to models without prior domain-specific fine-tuning; therefore, our findings may not reflect the full potential of LLMs further trained or adapted specifically for systematic review tasks. Finally, our search was restricted to English-language publications, which may have introduced language bias and led to the omission of relevant non-English studies. In addition, we relied solely on free-text terms in titles, abstracts, and keywords, and did not use controlled vocabulary (eg, MeSH/Emtree), which may have reduced the sensitivity of the search and resulted in missed studies. However, reference lists of included trials were also hand-searched for additional eligible studies, which may have partially mitigated the risk of missing relevant evidence. Given that this review provides some of the first evidence on the use of AI in a rare genetic disorder with a relatively small evidence base, future research should replicate similar AI-assisted workflows in scientific fields with larger and more complex datasets to determine how well AI performs in reviews involving larger evidence bases.

Implications and Conclusions

This review is distinctive in that it both synthesizes the fragmented trial evidence on emerging biologics in OI and prospectively evaluates an LLM-assisted workflow that combines screening and risk-of-bias assessment against consensus human judgments. Unlike prior reviews that focus either on pharmacologic efficacy or AI methods, we provide a unified, quantitative evaluation of both clinical effects and AI performance within a real-world rare-disease review. Taken together, our findings suggest that biologics should remain largely confined to clinical trials until fracture and safety outcomes are better established, while LLM-based tools can be used to reduce reviewer workload under structured human oversight and to inform the integration of LLM-assisted evidence surveillance into “living” rare-disease guidelines (eg, automated literature monitoring followed by targeted human appraisal). At a policy level, these findings highlight the need for governance frameworks that define minimum performance standards, transparency requirements, and human-oversight safeguards for AI-driven updates to rare-disease clinical guidance.

In conclusion, denosumab and setrusumab show potential efficacy in improving aBMD across age groups in patients with common COL1-related OI types (I-IV), yet evidence for fracture reduction remains inconclusive. Integrating biologics into clinical practice will require more rigorous, long-term RCTs, especially in children. LLM-based tools can improve evidence synthesis in rare diseases by integrating fragmented and heterogeneous clinical trial data. Refined prompts that support context-aware reasoning are essential for trustworthy AI, enabling LLMs to automate rule-based tasks under instruction-based workflows.